Apparatus and method for estimating uncertainty of image coordinate

ABSTRACT

An apparatus for estimating an uncertainty includes a processor configured to receive a first tracking coordinate corresponding to a reference coordinate, the reference coordinate being included in first image data acquired by a camera sensor, image data acquired after the first image data; acquire, based on motion data acquired from a motion sensor and a depth value of the first image data, a second tracking coordinate corresponding to the reference coordinate, the second tracking coordinate including a motion-based tracking coordinate in the second image data; calculate a target coordinate distribution in the second image data based on the first tracking coordinate and the second tracking coordinate; acquire an estimated target coordinate and an uncertainty of the estimated target coordinate based on the calculated target coordinate distribution; and update the first tracking coordinate based on the estimated target coordinate.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on and claims priority under 35 U.S.C. § 119to Korean Patent Applications No. 10-2021-0098529, filed on Jul. 27,2021, and No. 10-2021-0127547, filed on Sep. 27, 2021, in the KoreanIntellectual Property Office, the disclosures of which are incorporatedby reference herein in their entireties.

BACKGROUND 1. Field

The disclosure relates to apparatuses and methods for estimating theuncertainty of an image coordinate in order to acquire a trackingcoordinate of image data.

2. Description of Related Art

In the fields of computer vision and robotics, visual odometry (VO) andsimultaneous localization and mapping (SLAM) technologies have beenstudied. In particular, these technologies may be applied to autonomousnavigation and augmented reality, which have become increasinglypopular.

For the feature-based visual odometry and SLAM technologies, an imagecoordinate tracking method may be applied in which continuous image datais acquired through a moving camera sensor and a relationship amongportions of the continuous image data is analyzed by tracking coordinatemovement in the acquired image data. In the image coordinate trackingmethod, a target coordinate in current image data is determined withrespect to a specific coordinate in past image data. In this case, thespecific coordinate in the past image data and the target coordinate inthe current image data may be substantially the same point.

In a general image coordinate tracking method, a coordinate in currentimage data having a peripheral image that is most similar to aperipheral image of a specific coordinate in the past image data may beassumed as being a target coordinate. Afterwards, pairs that are clearlydistinguished from a plurality of pairs formed by a specific coordinatein the past image data and a target coordinate in the current image dataare estimated as indicate a tracking failure, and the tracking method isrepeated to improve the reliability thereof.

SUMMARY

In tracking a coordinate movement by acquiring continuous image datafrom a moving camera sensor, the tracking result of the moving camerasensor may vary greatly depending on a change in visual characteristics.For example, when the motion of a camera sensor is large, deteriorationof visual features, such as motion blur, illumination change, andocclusion, may be caused. In particular, when a motion blur is caused, aplurality of coordinates similar to a specific coordinate in past imagedata may be distributed in a direction in which the motion blur iscaused, which makes it difficult to estimate a target coordinate. Also,error occurrence may be accumulated due to difficulty in estimating thetarget coordinate, and accordingly, the accuracy of image coordinatetracking may be greatly reduced.

Accordingly, provided are apparatuses for estimating a target coordinateby using a motion sensor as well as a camera sensor and acquiringuncertainty of the estimated target coordinate, and operating methodsthereof.

The problems to be solved through the embodiments of the disclosure arenot limited to the problems described above, and problems not mentionedare clearly understood by those of ordinary skill in the art to whichthe embodiments belong from the disclosure and the accompanyingdrawings.

Additional aspects will be set forth in part in the description whichfollows and, in part, will be apparent from the description, or may belearned by practice of embodiments of the disclosure.

In accordance with an aspect of the disclosure, an apparatus forestimating an uncertainty includes a processor configured to estimate anuncertainty of an image coordinate by executing at least one program,wherein the processor is further configured to receive a first trackingcoordinate that corresponds to a reference coordinate, the referencecoordinate being included in first image data acquired by a camerasensor, wherein the first tracking coordinate includes an image-basedtracking coordinate in second image data acquired after the first imagedata; acquire, based on motion data acquired from a motion sensor and adepth value of the first image data, a second tracking coordinate thatcorresponds to the reference coordinate, wherein the second trackingcoordinate includes a motion-based tracking coordinate in the secondimage data; calculate a target coordinate distribution in the secondimage data based on the first tracking coordinate and the secondtracking coordinate; acquire an estimated target coordinate and anuncertainty of the estimated target coordinate based on the calculatedtarget coordinate distribution; and update the first tracking coordinatebased on the estimated target coordinate.

The processor may be further configured to acquire a firstthree-dimensional (3D) coordinate with respect to the referencecoordinate based on the depth value of the first image data; convert thefirst 3D coordinate into a second 3D coordinate in the second image databased on the motion data; and acquire the second tracking coordinate byprojecting the second 3D coordinate.

The processor may be further configured to, for each of a plurality ofcoordinates included in the second image data, calculate a weightedaverage of a first distance value and a second distance value, the firstdistance value being between the coordinate and the first trackingcoordinate and the second distance value being between the coordinateand the second tracking coordinate; for each of the plurality ofcoordinates, calculate a prior probability distribution based on thecalculated weighted average; determine a portion of the priorprobability distribution having a prior probability greater than orequal to a threshold value as a candidate coordinate group; andcalculate the target coordinate distribution based on the candidatecoordinate group.

The processor may be further configured to, for each of the plurality ofcoordinates, when a value included in the motion data is less than apreset value, set a weight parameter corresponding to the first distancevalue to be greater than a weight parameter corresponding to the secondtracking coordinate.

The processor may be further configured to calculate an image-baseddistribution based on image similarity with the reference coordinate forthe determined candidate coordinate group.

The processor may be further configured to calculate the targetcoordinate distribution based on the prior probability distribution andthe image-based distribution.

The processor may be further configured to transmit the updated firsttracking coordinate and the uncertainty of the estimated targetcoordinate to an external apparatus.

The processor may be further configured to estimate a target coordinateby calculating an average of the target coordinate distribution, andestimate the uncertainty of the estimated target coordinate bycalculating a covariance matrix for the estimated target coordinate.

In accordance with an aspect of the disclosure, a method of estimatingan uncertainty includes receiving a first tracking coordinate thatcorresponds to a reference coordinate, the reference coordinate beingincluded in first image data acquired by a camera sensor, wherein thefirst tracking coordinate includes an image-based tracking coordinate insecond image data acquired after the first image data; acquiring, basedon motion data acquired from a motion sensor and a depth value of thefirst image data, a second tracking coordinate that corresponds to thereference coordinate, wherein the second tracking coordinate includes amotion-based tracking coordinate in the second image data; calculating atarget coordinate distribution in the second image data based on thefirst tracking coordinate and the second tracking coordinate; acquiringan estimated target coordinate and an uncertainty of the estimatedtarget coordinate based on the calculated target coordinatedistribution; and updating the first tracking coordinate based on theestimated target coordinate.

The acquiring of the second tracking coordinate may include acquiring afirst 3D coordinate with respect to the reference coordinate based onthe depth value of the first image data; converting the 3D coordinateinto a second 3D coordinate in the second image data based on the motiondata; and acquiring the second tracking coordinate by projecting theconverted second 3D coordinate.

The method may further include, for each of a plurality of coordinatesincluded in the second image data, calculating a weighted average of afirst distance value and a second distance value, the first distancevalue being between the coordinate and the first tracking coordinate andthe second distance value being between the coordinate and the secondtracking coordinate; for each of the plurality of coordinates,calculating a prior probability distribution based on the calculatedweighted average; determining, from among the prior probabilitydistributions, coordinates from among the plurality of coordinateshaving a prior probability greater than or equal to a threshold value asa candidate coordinate group for the estimated target coordinate; andcalculating the target coordinate distribution based on the candidatecoordinate group.

The method may further include calculating an image-based distributionfor the determined candidate coordinate group based on image similaritywith the reference coordinate.

The method may further include calculating the target coordinatedistribution based on the prior probability distribution and theimage-based distribution.

The method may further include transmitting the updated first trackingcoordinate and the uncertainty of the estimated target coordinate to anexternal apparatus.

The method may further include estimating a target coordinate bycalculating an average of the target coordinate distribution, andestimating the uncertainty of the estimated target coordinate bycalculating a covariance matrix for the estimated target coordinate.

In accordance with an aspect of the disclosure, an electronic apparatusfor performing a simultaneous localization and mapping (SLAM) operationincludes a camera sensor configured to acquire image data about asurrounding environment; a motion sensor configured to acquire motiondata by detecting a rotation and a movement of the electronic apparatus;and a processor electrically connected to the camera sensor and themotion sensor, wherein the processor is configured to receive a firsttracking coordinate that corresponds to a reference coordinate, thereference coordinate being included in first image data acquired by thecamera sensor, the first tracking coordinate including an image-basedtracking coordinate in second image data acquired after the first imagedata; acquire, based on the motion data acquired from the motion sensorand a depth value of the first image data, a second tracking coordinatethat corresponds to the reference coordinate and is a motion-basedtracking coordinate in the second image data; calculate a targetcoordinate distribution in the second image data based on the firsttracking coordinate and the second tracking coordinate; acquire anestimated target coordinate and an uncertainty of the estimated targetcoordinate based on the calculated target coordinate distribution; andupdate the first tracking coordinate based on the estimated targetcoordinate.

The processor may be further configured to acquire a first 3D coordinatewith respect to the reference coordinate based on the depth value of thefirst image data; convert the first 3D coordinate into a second 3Dcoordinate in the second image data based on the motion data; andacquire the second tracking coordinate by projecting the convertedsecond 3D coordinate.

The electronic apparatus may further include a back-end processorconfigured to calculate a rotation angle and a position of the camerasensor, wherein the processor is further configured to transmit theupdated first tracking coordinate and the uncertainty of the estimatedtarget coordinate to the back-end processor.

The processor may be further configured to estimate a target coordinateby calculating an average of the target coordinate distribution, andestimate the uncertainty of the estimated target coordinate bycalculating a covariance matrix with respect to the estimated targetcoordinate.

The processor may be further configured to display an imagecorresponding to the updated first tracking coordinate.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features, and advantages of certainembodiments of the disclosure will be more apparent from the followingdescription taken in conjunction with the accompanying drawings, inwhich:

FIG. 1 is a block diagram of an electronic apparatus according to anembodiment;

FIG. 2 is an example diagram for explaining a conventional method oftracking an image coordinate;

FIG. 3 is a flowchart for explaining a method used by an apparatus toestimate an uncertainty of tracking a coordinate according to anembodiment;

FIG. 4 is an example diagram for explaining a method used by anelectronic apparatus to acquire a motion-based tracking coordinateaccording to an embodiment;

FIG. 5 is a detailed flowchart for explaining a method used by anapparatus to estimate an uncertainty of tracking a coordinate accordingto an embodiment;

FIG. 6 is an example diagram for explaining a method used by anapparatus to determine a grid to sample image data, according to anembodiment; and

FIG. 7 is a perspective view of an electronic apparatus according to anembodiment.

DETAILED DESCRIPTION

Reference will now be made in detail to embodiments, examples of whichare illustrated in the accompanying drawings, wherein like referencenumerals refer to like elements throughout. In this regard, embodimentsmay have different forms and should not be construed as being limited tothe descriptions set forth herein. Accordingly, embodiments are merelydescribed below, by referring to the figures, to explain aspects. Asused herein, the term “and/or” includes any and all combinations of oneor more of the associated listed items. Expressions such as “at leastone of,” when preceding a list of elements, modify the entire list ofelements and do not modify the individual elements of the list.

Terminologies used herein are selected as commonly used by those ofordinary skill in the art in consideration of functions of the currentembodiment, but may vary according to the technical intention,precedents, or a disclosure of a new technology. Also, in particularcases, some terms are arbitrarily selected by the applicant, and in thiscase, the meanings of the terms will be described in detail atcorresponding parts of the specification. Accordingly, the terms used inthe specification should be defined not by simply the names of the termsbut based on the meaning and contents of the whole specification.

In the descriptions of the embodiments, it will be understood that, whenan element is referred to as being connected to another element, it mayinclude electrically connected when the element is directly connected tothe other element and when the element is indirectly connected to theother element by intervening a constituent element. Also, it should beunderstood that, when a part “comprises” or “includes” a constituentelement in the specification, unless otherwise defined, it is notexcluding other elements but may further include other elements.

The term “comprises” or “includes” used in the embodiments should not beconstrued as necessarily including various constituent elements andvarious operations described in the specification, and also should notbe construed that portions of the constituent elements or operations ofthe various constituent elements and various operations may not beincluded or additional constituent elements and operations may furtherbe included.

It will be understood that, although the terms ‘first’, ‘second’, etc.may be used herein to describe various constituent elements, theseconstituent elements should not be limited by these terms. These termsare only used to distinguish one constituent element from another.

Also, the ‘world coordinate system’ used in the disclosure may denote athree-dimensional coordinate system set based on the real world.

The descriptions of the embodiments should not be interpreted aslimiting the scope of right, and embodiments that are readily inferredfrom the detailed descriptions and embodiments by those of ordinaryskill in the art will be construed as being included in the disclosure.Reference will now be made in detail to embodiments, examples of whichare illustrated in the accompanying drawings.

FIG. 1 is a block diagram of an electronic apparatus 100 according to anembodiment.

Referring to FIG. 1 , the electronic apparatus 100 according to anembodiment includes an uncertainty estimation apparatus 110, a camerasensor 120, a motion sensor 130, a front-end processor 140, and aback-end processor 150.

In an embodiment, the electronic apparatus 100 may be an electronicapparatus configured to recognize a current pose of the electronicapparatus 100 in a three-dimensional space based on image informationacquired from the camera sensor 120 and to model a surroundingenvironment. The electronic apparatus 100 may be an electronic apparatusto which visual odometry (VO), visual simultaneous localization andmapping (V-SLAM), and/or visual-inertial odometry (VIO) technology areapplied. For example, as the VO, V-SLAM, and/or VIO technologies areapplied, the electronic apparatus 100 may be an electronic apparatusconfigured to provide autonomous flying drones, robots, autonomousvehicles, virtual reality and/or augmented reality (e.g., smart glass).However, the disclosure is not limited thereto, and the electronicapparatus 100 may include various electronic apparatuses to which theabove technologies are applied.

In an embodiment, the camera sensor 120 may acquire image data about asurrounding environment. For example, the camera sensor 120 maycorrespond to an omnidirectional camera, a stereo camera, or a monocamera. In an embodiment, the camera sensor 120 may acquire image dataincluding 3D information of various objects (e.g., static and/or dynamicobjects) in a dynamic environment. In this case, the image data acquiredby the camera sensor 120 may include not only image data of each pixelbut also depth data of each pixel.

In an embodiment, the motion sensor 130 may acquire motion data bydetecting rotation (e.g., a rotation angle) and translation of theelectronic apparatus 100. For example, the motion sensor 130 may be aninertial measurement unit (IMU) including an acceleration sensor and/ora gyro sensor.

In an embodiment, the front-end processor 140 may process data receivedfrom the camera sensor 120 and the motion sensor 130. For example, thefront-end processor 140 may acquire data for tracking an imagecoordinate by processing data received from the camera sensor 120 andthe motion sensor 130.

In an embodiment, the front-end processor 140 may analyze image datareceived from the camera sensor 120. For example, the front-endprocessor 140 may acquire a visual correspondence by analyzing the imagedata. In the disclosure, a ‘visual correspondence’ may denote arelationship between two-dimensional coordinates projected for each ofimage data in which a specific three-dimensional coordinate iscontinuously acquired.

In an embodiment, the front-end processor 140 may acquire a trackingcoordinate through an image coordinate tracking algorithm. That is, thefront-end processor 140 may acquire a visual correspondence between aplurality of image data received from the camera sensor 120 by using animage coordinate tracking algorithm, and may acquire the trackingcoordinate based on the acquired visual correspondence.

For example, the front-end processor 140 may receive first image datafrom the camera sensor 120. In this case, the front-end processor 140may set a reference coordinate (x_(r)) of the first image data, and areference coordinate may be set to correspond to a key point. Afterreceiving the first image data, the front-end processor 140 may receivesecond image data from the camera sensor 120. The front-end processor140 may acquire a visual correspondence based on the image similarity ofthe first image data and the second image data. In this case, thefront-end processor 140 may calculate an image similarity between thereference coordinate (x_(r)) set in the first image data and anarbitrary coordinate (x_(r)) in the second image data, and the imagesimilarity may be calculated through Equation 1.

$\begin{matrix}{{{Image}{similarity}} = {- {\sqrt{\frac{1}{\left( {{2N} + 1} \right)^{2}}{\sum{\text{?}\left( {{I{\text{?}\left\lbrack {x_{r} + \left( {x,y} \right)} \right\rbrack}} - {I_{2}\left\lbrack {x + \left( {x,y} \right)} \right\rbrack}} \right)^{2}}}}.}}} & \left\lbrack {{Equatio}n1} \right\rbrack\end{matrix}$ ?indicates text missing or illegible when filed

That is, the image similarity may be calculated as a root mean squareerror (RMSE) of pixel values in the square of (2N+1)×(2N+1) around acoordinate. The front-end processor 140 may determine a coordinate atwhich the image similarity is maximized as a first tracking coordinate(x_(r)) through the Kanade-Lucas-Tomasi (KLT) algorithm. In this case,the ‘first tracking coordinate (x_(r))’ may denote an image-basedtracking coordinate.

In an embodiment, the front-end processor 140 may estimate an inertialpose based on motion data received from the motion sensor 130. Forexample, when the mion sensor 130 is a 6 DoF inertial measurementdevice, the front-end processor 140 may receive 3-axis linearacceleration and 3-axis angular velocity from the motion sensor 130. Thefront-end processor 140 may integrate data about the linear accelerationand the angular velocity by estimating an initial velocity and a biasvalue of a sensor. In this case, as the front-end processor 140estimates and integrates the initial velocity and the bias value of thesensor, accumulation of errors may be prevented. The front-end processor140 may estimate an inertial pose (e.g., (R, t) by integrating the databy estimating an initial velocity and a bias value of the sensor.

In an embodiment, the front-end processor 140 may estimate a depth value(d) of image data based on the first tracking coordinate (x_(r)), whichis an image-based tracking coordinate, and motion data. For example, thefront-end processor 140 may estimate a depth value (d) of the image databy performing triangulation based on the first tracking coordinate(x_(r)) and the motion data.

In an embodiment, the front-end processor 140 may transmit data aboutthe first tracking coordinate (x_(r)) the inertia pose (R, t), and thedepth value (d) of the image data to the uncertainty estimationapparatus 110.

With further reference to FIG. 1 , in an embodiment, the uncertaintyestimation apparatus 110 may include at least one processor 115. Forexample, the processor 115 may include an embedded processor, amicro-processor, a hardware control logic, a finite state machine (FSM),or a digital signal processing (DSP) or a combination thereof.

In the disclosure, the ‘uncertainty estimation apparatus’ refers to anapparatus for estimating uncertainty of a visual correspondence, andspecifically, may refer to an apparatus for estimating a probabilitydifference between a visual correspondence and an actual correspondence.Also, the uncertainty estimation apparatus 110 may include at least oneprocessor 115, but is not limited thereto, and the at least oneprocessor 115 may itself be the uncertainty estimation apparatus 110.

In one embodiment, the processor 115 may determine a second trackingcoordinate (x_(m)) based on the first tracking coordinate (x_(r)) theinertial pose (R, t) and the depth value (d) of the image data receivedfrom the front-end processor 140. In this case, the ‘second trackingcoordinate (x_(m))’ may denote a motion-based tracking coordinate. Adetailed description of a method for the processor 115 to determine thesecond tracking coordinate (x_(m))will be described below with referenceto FIG. 4 .

In an embodiment, after determining the second tracking coordinate(x_(m)) the processor 115 may calculate a target coordinate distributionin second image data through a series of operations. For example, theprocessor 115 may finally calculate a target coordinate distribution inthe second image data through performing an operation of calculating aprior probability distribution (operation 1), an operation ofdetermining a candidate coordinate group (operation 2), and an operationof calculating an image-based distribution (operation 3). A detaileddescription for the processor 115 to calculate the target coordinatedistribution in the second image data will be described below withreference to FIG. 5 .

In an embodiment, the processor 115 may acquire a plurality of targetcoordinate distributions and a plurality of grids (x_(n)) by repeatingthe process of calculating the target coordinate distribution for theplurality of second tracking coordinates (x_(m) ^(n)) The processor 115may acquire an estimated target coordinate (μ) and the uncertainty (Σ)of the estimated target coordinate based on the plurality of targetcoordinate distributions and the plurality of grids (X_(n)). Forexample, the processor 115 may acquire the estimated target coordinate(μ) by calculating an average for a plurality of target coordinatedistributions, and acquire the uncertainty (Σ) of the estimated targetcoordinate by calculating a covariance matrix for the estimated targetcoordinate. A detailed description of a method by which the processor115 acquires the estimated target coordinate (μ) and the uncertainty (Σ)of the estimated target coordinate will be described below withreference to FIG. 5 .

In an embodiment, the processor 115 may update the first trackingcoordinate (x_(r)) received from the front-end processor 140 based onthe acquired estimated target coordinate (μ) That is, the updated firsttracking coordinate may denote the same coordinate as the acquiredestimated target coordinate (μ) Thereafter, the processor 115 maytransmit the updated first tracking coordinate (μ) and uncertainty (Σ)to the back-end processor 150 (e.g., to an external apparatus). Forexample, the processor 115 may transmit the updated first trackingcoordinate (μ) to be displayed on a display or may transmit the updatedfirst tracking coordinate (μ) to another device for use in anapplication relating to autonomous navigation or augmented reality.

In an embodiment, the back-end processor 150 may receive the firsttracking coordinate (μ)and the uncertainty (Σ)updated from theuncertainty estimation apparatus 110, and receive an estimated inertialpose (R, t) from the front-end processor 140. The back-end processor 150may calculate a pose (e.g., a position and an orientation) of the camerasensor 120 based on the received data. Also, the back-end processor 150may transmit a result of performing optimization through a Kalman filterand/or bundle adjustment to the uncertainty estimation apparatus 110.For example, the back-end processor 150 may transmit the result data ofcalculating and optimizing a pose of the camera sensor 120 (e.g., adepth value (d) of the second image data) to the uncertainty estimationapparatus 110.

FIG. 2 is an example diagram for explaining a conventional method oftracking an image coordinate.

Referring to FIG. 2 , according to the movement of a user wearing anelectronic apparatus 200, the electronic apparatus 200 may output imagedata to which the user's movement is reflected through a display. Forexample, when a user's pose changes from a first pose 210 a to a secondpose 210 b according to the movement of the user wearing the electronicapparatus 200, the electronic apparatus 200 may output image data towhich the change of pose is reflected through the display. In this case,the first pose 210 a may include the position and orientation of theuser who wears the electronic apparatus 200 at the first time point. Thesecond pose 210 b may include the position and orientation of the userwho wears the electronic apparatus 200 at a second time point which islater than the first time point.

In an embodiment, the electronic apparatus 200 may track an imagecoordinate through a conventional image coordinate tracking algorithm.That is, the electronic apparatus 200 may track the image coordinateonly through a visual correspondence between continuous image dataacquired through a camera (e.g., the camera sensor 120 of FIG. 1 ).

For example, the electronic apparatus 200 may set a reference coordinate220 of first image data 205 at a first time point. In this case, thereference coordinate 220 may refer to a coordinate that is easilyidentified even if a viewpoint of the camera or a surroundingenvironment changes. The reference coordinate 220 may be set through akey point extraction method (e.g., Harris Corner method).

Thereafter, as the user wearing the electronic apparatus 200 moves, theelectronic apparatus 200 may output second image data 215 at the secondtime point. At this time, if the user's movement is large, motion blurmay be caused in the second image data 215, and the first image data 205and the second image data 215 are flattened due to a difference in theimage, and thus, the identification of the coordinate may be difficult.Although the electronic apparatus 200 samples an image differenceoccurred between the first image data 205 and the second image data 215,because a plurality of coordinates similar to the reference coordinate220 are distributed in the second image data 215, the effect of thesampling may be reduced. Accordingly, if the effect of sampling on theimage difference between the first image data 205 and the second imagedata 215 is reduced, the probability distribution for a final targetcoordinate may be inaccurate, and as a result, the uncertaintycalculation may be disturbed.

FIG. 3 is a flowchart for explaining a method used by an apparatus toestimate an uncertainty of tracking a coordinate according to anembodiment.

Referring to FIG. 3 , in operation 301, a processor (e.g., the processor115 of FIG. 1 ) of an uncertainty estimation apparatus (e.g., theuncertainty estimation apparatus 110 of FIG. 1 ) may receive a firsttracking coordinate (x_(r))that is an image-based tracking coordinate inthe second image data. The first tracking coordinate (x_(r)) maycorrespond to a reference coordinate (x_(r))of first image data.

In the disclosure, ‘second image data’ may denote image data acquiredafter ‘first image data’, and ‘first image data’ and ‘second image data’may each denote continuous image data. In addition, the uncertaintyestimation apparatus in the present disclosure may estimate anuncertainty of a target coordinate with respect to the ‘second imagedata’ having a large image difference from the ‘first image data’ due tomotion blur caused by a large movement of a user, parallax, or lightingchange.

In an embodiment, the processor 115 may receive the first trackingcoordinate (x_(r)) that is an image-based tracking coordinate in thesecond image data from the front-end processor 140. The front-endprocessor 140 may acquire a visual correspondence based on an imagesimilarity of the first image data and the second image data, and maydetermine the first tracking coordinate (x_(r)) based on the acquiredvisual correspondence. For example, the front-end processor 140 maycalculate the image similarity between the reference coordinate (x_(r))set in the first image data and an arbitrary coordinate (x) in thesecond image data, and may determine a coordinate at which the imagesimilarity is maximized as the first tracking coordinate (x_(r)) throughthe KLT algorithm.

According to an embodiment, in operation 303, the processor 115 mayacquire a second tracking coordinate (x_(m))that is a motion-basedtracking coordinate in the second image data. In an embodiment, theprocessor 115 may acquire the second tracking coordinate (x_(m)) basedon motion data received from the front-end processor 140 and a depthvalue (d) of the first image data. In this case, the ‘motion data’ maydenote an inertial pose (R, t) estimated from the front-end processor140, and the ‘depth value (d) of the first image data’ may denote adepth value (d) estimated by the front-end processor 140 throughtriangulation.

For example, the processor 115 may convert the reference coordinate(x_(r))in the first image data into a three-dimensional (3D) coordinate(e.g., a first 3D coordinate) based on the depth value (d) of the firstimage data. With respect to the converted 3D coordinate, the processor115 may convert the converted 3D coordinate into a 3D coordinate in thesecond image data (e.g., a second 3D coordinate) by applying an inertialpose (R, t) received from the front-end processor 140. In addition, theprocessor 115 may acquire a second tracking coordinate (x_(m)) byprojecting the 3D coordinate in the second image data.

According to an embodiment, in operation 305, the processor 115 maycalculate a target coordinate distribution in the second image data. Inthe disclosure, the ‘target coordinate distribution’ may denote aprobability distribution of a coordinate to which the referencecoordinate (x_(r)) of the first image data substantially correspondsamong the coordinates of the second image data.

In an embodiment, the processor 115 may calculate a target coordinatedistribution based on the first tracking coordinate (x_(r))that is animage-based tracking coordinate and the second tracking coordinate(x_(m))that is a motion-based tracking coordinate in the second imagedata. For example, the processor 115 may finally calculate a targetcoordinate distribution in the second image data through performing anoperation of calculating a prior probability distribution (operation 1),an operation of determining a candidate coordinate group (operation 2),and an operation of calculating an image-based distribution (operation3). A detailed description about the method of calculating a targetcoordinate distribution in the second image data by the processor 115will be described below with reference to FIG. 5 .

According to an embodiment, in operation 307, the processor 115 mayacquire an estimated target coordinate and the uncertainty of theestimated target coordinate. In the disclosure, the ‘target coordinate’may denote a coordinate substantially corresponding to the referencecoordinate of the first image data, and the ‘estimated targetcoordinate’ may denote a coordinate estimated to correspond to thereference coordinate of the first image data. In this case, the‘estimated target coordinate’ may be an image-based tracking coordinate(e.g., the first tracking coordinate (x_(r))).

In an embodiment, the processor 115 may acquire the estimated targetcoordinate and the uncertainty of the estimated target coordinate basedon the target coordinate distribution calculated in operation 305. Forexample, the processor 115 may acquire a plurality of target coordinatedistributions and a plurality of grids (X_(n)) by repeating the processof calculating the target coordinate distribution with respect to theplurality of second tracking coordinates (x_(m) ^(n)) The processor 115may acquire the estimated target coordinate (μ)and the uncertainty (Σ)of the estimated target coordinate based on the plurality of targetcoordinate distributions and the plurality of grids (X_(n)). Forexample, the processor 115 may acquire an estimated target coordinate(μ) by calculating an average for a plurality of target coordinatedistributions, and may acquire the uncertainty (Σ) of the estimatedtarget coordinate by calculating a covariance matrix for the estimatedtarget coordinate.

According to an embodiment, in operation 309, the processor 115 mayupdate the first tracking coordinate based on the estimated targetcoordinate (μ). For example, the processor 115 may update and set theestimated target coordinate (μ), which is an average of a plurality oftarget coordinate distributions, as the first tracking coordinate thatis an image-based tracking coordinate of the second image data.

FIG. 4 is an example diagram for explaining a method for an electronicapparatus to acquire a motion-based tracking coordinate according to anembodiment.

Referring to FIG. 4 , an electronic apparatus (e.g., the electronicapparatus 100 of FIG. 1 ) may output image data through a display. Forexample, the electronic apparatus 100 may output first image data 405including a cube-shaped object.

In an embodiment, a front-end processor (e.g., the front-end processor140 of FIG. 1 ) may set a reference coordinate (x_(r)) 420 of the firstimage data 405. For example, the front-end processor 140 may set afeature point (e.g., a corner point of the cube-shaped object) of thefirst image data 405 as the reference coordinate (x_(r)) 420.

In an embodiment, the front-end processor 140 may determine a firsttracking coordinate (x_(r)) 430 that is an image-based trackingcoordinate based on the image similarity of the first image data 405 andthe second image data 415.

In an embodiment, the front-end processor 140 may estimate an inertialpose based on motion data received from a motion sensor (e.g., themotion sensor 130 of FIG. 1 ). For example, the front-end processor 140may integrate data on a linear acceleration and angular velocity byestimating initial velocity and bias values of the motion sensor 130,and may estimate an inertial pose (R, t) as a result of the integration.

In an embodiment, the front-end processor 140 may acquire a depth value(d) 422 of the first image data 405 from a camera sensor (e.g., thecamera sensor 120 of FIG. 1 ). For example, when the camera sensor 120recognizes a 3D depth value of image data through a stereoscopy-typemethod, a time-of-flight (ToF) method, a structured pattern method, orthe like, the front-end processor 140 may acquire the depth value (d)422 of the first image data 405 from the camera sensor 120. In anembodiment, the front-end processor 140 may estimate the depth value (d)of the image data by performing triangulation based on the firsttracking coordinate (x_(r)) and the motion data.

In an embodiment, the front-end processor 140 may transmit data aboutthe first tracking coordinate (x_(r)) the inertial pose (R, t), and thedepth value (d)of the image data to the uncertainty estimation apparatus110.

In an embodiment, a processor (e.g., the processor 115 of FIG. 1 ) of anuncertainty estimation apparatus (e.g., the uncertainty estimationapparatus 110 of FIG. 1 ) may acquire a second tracking coordinate(x_(m))that is a motion-based tracking coordinate of the second imagedata 415 through Equation 2. The processor 115 may calculate Equation 2based on data received from the front-end processor 140.

x _(m) =π[Rπ ^(−i) [x _(r) ,d]+t],   Equation 2]

At this time, in Equation 2, π[⋅*] is a projection function thatreceives a 3D coordinate and outputs a coordinate (that is,two-dimensional (2D) coordinate) in an image, and π⁻¹[*,d] correspondsto a back-projection function that receives a 2D coordinate and a depthvalue in an image and outputs a 3D coordinate. The processor 115 mayacquire a 3D coordinate 424 for the reference coordinate (x_(r)) 420 bycalculating the reference coordinate (x_(r)) 420 of the first image data405 according to a back-projection function π⁻¹[x_(r), d] based on thedepth value (d) 422.

The processor 115 may acquire the 3D coordinate in the second image data415 by applying an inertial pose (R, t) to the acquired 3D coordinate424. That is, the processor 115 may perform a multiplication operationfor multiplying the back-projection function π⁻¹[x_(r), d], which is the3D coordinate 424 of the reference coordinate (x_(r)) 420, by R, whichis a rotation value of the electronic device 100. Also, the processor115 may perform a sum operation on a value Rπ⁻¹[x_(r), d], which is aresult of the multiplication operation and t, which is a translationvalue of the electronic device 100. The processor 115 may determine avalue Rπ⁻¹[x_(r)d]+t, which is a result of performing the sum operationas the 3D coordinate in the second image data 415.

The processor 115 may acquire a second tracking coordinate (x_(m)) 440,which is a motion-based tracking coordinate, by calculating the 3Dcoordinate in the second image data 415 according to the projectionfunction π[Rπ⁻¹[x_(r), d]+t].

FIG. 5 is a detailed flowchart for explaining a method for an apparatusto estimate the uncertainty of tracking a coordinate according to anembodiment.

Referring to FIG. 5 , each of the camera sensor 120 and the motionsensor 130 may transmit acquired data to the front-end processor 140.For example, in operation 500, the camera sensor 120 that has acquiredimage data for a surrounding environment may transmit the image data tothe front-end processor 140. As another example, in operation 505, themotion sensor 130 that acquires motion data by detecting rotation andtranslation of an electronic apparatus (e.g., the electronic apparatus100 of FIG. 1 ) may transmit the motion data to the front-end processor140.

FIG. 5 shows an embodiment in which the camera sensor 120 and the motionsensor 130 sequentially transmit the acquired data, but the disclosureis not limited thereto. Each of the camera sensor 120 and the motionsensor 130 may transmit the acquired data to the front-end processor 140in parallel.

In an embodiment, the front-end processor 140 may process image data andmotion data received from the camera sensor 120 and the motion sensor130. For example, the front-end processor 140 may acquire a firsttracking coordinate (x_(r)) which is an image-based tracking coordinate,based on the image data received from the camera sensor 120. Thefront-end processor 140 may estimate an inertia pose (R, t) of theelectronic apparatus 100 based on the motion data received from themotion sensor 130. Also, the front-end processor 140 may estimate adepth value (d) of the image data based on the first tracking coordinate(x_(r)) which is an image-based tracking coordinate, and the motiondata.

According to an embodiment, in operation 510, the front-end processor140 may transmit data about the first tracking coordinate (x_(r)) theinertial pose (R, t), and the depth value (d) of the image data to theprocessor 115 of an uncertainty estimation apparatus (e.g., theuncertainty estimation apparatus 110 of FIG. 1 ).

According to an embodiment, in operation 515, the processor 115 mayacquire a second tracking coordinate (x_(m)) that is a motion-basedtracking coordinate of the second image data. For example, the processor115 may acquire the second tracking coordinate (x_(m)) based on data ofthe reference coordinate (x_(r)), the inertial pose (R, t), and thedepth value (d) of the image data that are received from the front-endprocessor 140.

In an embodiment, the processor 115 may calculate a target coordinatedistribution in the second image data through a series of operationsafter acquiring the second tracking coordinate (x_(m)).

For example, in operation 520, the processor 115 may calculate a priorprobability distribution. In the disclosure, the ‘prior probabilitydistribution’ may denote a probability distribution for an arbitrarycoordinate x sampled through the image-based tracking coordinate and themotion-based tracking coordinate. The prior probability distribution maydenote a distance-based probability of an arbitrary coordinate x. Theprocessor 115 may calculate a prior probability distribution withrespect to the arbitrary coordinate x through Equation 3.

$\begin{matrix}{{q\left( {x{❘{x_{v},x_{m}}}} \right)} = {\frac{1}{Z_{\alpha,\beta}}{\exp\left\lbrack {- {\alpha\left( {{\beta{❘{x - x_{v}}❘}} + {\left( {1 - \beta} \right){❘{x - x_{m}}❘}}} \right)}} \right\rbrack}}} & \left\lbrack {{Equation}3} \right\rbrack\end{matrix}$

That is, the processor 115 may calculate a prior probabilitydistribution by using a weighted average of |x−x_(v)|, which is adistance value (e.g., a first distance value) between the arbitrarycoordinate x and the image-based tracking coordinate (x_(r)) and|x−x_(m)|, which is a distance value (e.g., a second distance value)between the arbitrary coordinate x and the motion-based trackingcoordinate (x_(m)).

In this case, β may be a weight parameter for balancing between theimage-based tracking coordinate (x_(r)) and the motion-based trackingcoordinate (x_(m))

In this case, β may be in a range of 0 to 1. For example, when a value(e.g., a value corresponding to the inertial pose (R, t)) included inthe motion data is less than a preset value, because informationacquired from the camera sensor 120 may be relatively more accurate, inorder to increase the importance of the image based tracking coordinate(x_(r)) relative to the motion-based tracking coordinate (x_(m)), thevalue of β may be increased. As the value of β increases, the importanceof the motion-based tracking coordinate (x_(m)) may decrease. α is ascale parameter, Z_(α,β) and may be a constant that is a normalizationfactor.

In an embodiment, the two parameters α and β may be adjusted by usingprobability density values of the image-based tracking coordinate(x_(v)) and the motion-based tracking coordinate (x_(m))as in Equations4 and 5.

q(x _(r) |x _(r) , x _(m))=s×q(x _(m) |x _(r) , x _(m))   [Equation 4]

q(x _(r) |x _(r) , x _(m))+q(x _(m) |x _(r) , x _(m))=h   [Equation 5]

In this case, s may be a parameter for adjusting the relative importanceof the image-based tracking coordinate (x_(r)) and the motion-basedtracking coordinate (x_(m)), and h may be a parameter for adjusting theheight of a prior probability distribution.

In operation 520, after calculating the prior probability distribution,the processor 115 may determine a candidate coordinate group inoperation 525. For example, the processor 115 may determine a portion ofthe prior probability distribution having a prior probability equal toor greater than a threshold value as a candidate coordinate group. Theprocessor 115 may select the smallest rectangular region including acontour line of the threshold value as the grid region (R). Accordingly,the selected grid region (R) may be determined differently according tothe shape of the prior probability distribution.

In operation 530, the processor 115 may calculate an image-baseddistribution. For example, the processor 115 may calculate animage-based distribution based on the image similarity with thereference coordinate for the grid region R of the candidate coordinategroup determined in operation 525. In this case, the image-baseddistribution may be calculated through Equation 6.

$\begin{matrix}{{p_{k}\left( {x{❘x_{r}}} \right)} = {\frac{1}{Z_{k}}{\exp\left\lbrack {k \times {image}{similarity}} \right\rbrack}}} & \left\lbrack {{Equation}6} \right\rbrack\end{matrix}$

In this case, k may be a scale factor, and Z_(k) may be a constant thatis a normalization factor for the grid region (R). In an embodiment, maybe a scale factor that makes an image-based distribution (p_(k)(x|x_(r))and the prior probability distribution) (q(x|x_(r), x_(m)) becomesimilar, and k may be determined by minimizing KL-divergence thatindicates a different degree of the two distributions.

In operation 535, the processor 115 may calculate a target coordinatedistribution. For example, the processor 115 may calculate the targetcoordinate distribution based on the image-based distribution and theprior probability distribution. In this case, the target coordinatedistribution may be calculated through Equation 7.

p _(λ)(x|x _(r) , x _(r) , x _(m))=(p _(k)(x|x _(r)))^(1−λ)(q(x|x _(r) ,x _(m)))²   [Equation 7]

The processor 115 may calculate a target coordinate distribution byusing a weighted geometric mean of an image-based distribution and aprior probability distribution. In this case, λ may be a weightparameter for balancing the image-based distribution and the priorprobability distribution, and may be in a range of 0 to 1.

The processor 115 may acquire a plurality of target coordinatedistributions and a plurality of grid regions (R_(n)) by performingoperations 520 to 535 with respect to a plurality of motion-basedtracking coordinates (e.g., x_(m) ¹, x_(m) ², . . . , x_(m) ^(n)).

In operation 540, the processor 115 may acquire an estimated targetcoordinate and the uncertainty of the estimated target coordinate. Forexample, the processor 115 may acquire the estimated target coordinate(μ) based on a plurality of target coordinate distributionsp_(λ)(x|x_(r), x_(r), x_(m) ^(n)), a weight corresponding to theaccuracy of each of the plurality of motion-based tracking coordinates,and a plurality of grid regions R_(n). The estimated target coordinate(μ) may be calculated through

Equation 8.

$\begin{matrix}{\mu = {\frac{1}{\sum_{n}{❘R_{n}❘}}{\sum_{n}{w_{n}{\sum_{x \in R_{n}}{{xp}_{\lambda}\left( {x{❘{x_{r},x_{v},x_{m}^{n}}}} \right)}}}}}} & \left\lbrack {{Equation}8} \right\rbrack\end{matrix}$

In addition, the processor 115 may acquire the uncertainty (Σ) of theestimated target coordinate based on the estimated target coordinate(μ). The uncertainty (Σ)of the estimated target coordinate may becalculated through Equation 9.

$\begin{matrix}{{\sum{= {\frac{1}{\sum_{n}{❘R_{n}❘}}{\sum_{n}{w_{n}{\sum_{x \in R_{n}}{\left( {x - \mu} \right)^{T}\left( {x - \mu} \right){p_{\lambda}\left( {x{❘{x_{r},x_{v},x_{m}^{n}}}} \right)}}}}}}}},} & \left\lbrack {{Equation}9} \right\rbrack\end{matrix}$

That is, the processor 115 may acquire the estimated target coordinate(μ) by calculating an average of a plurality of target coordinatedistributions, and may acquire the uncertainty (Σ)of the estimatedtarget coordinate by calculating a covariance matrix for the estimatedtarget coordinate.

FIG. 6 is an example diagram for explaining a method for an apparatus todetermine a grid to sample image data, according to an embodiment.

Referring to FIG. 6 , a processor (e.g., the processor 115 of FIG. 1 )may select as a grid region (R) a portion of the prior probabilitydistribution having a prior probability greater than or equal to athreshold value c from among the prior probability distributionscalculated in operation 520 of FIG. 5 .

In an embodiment, when an image-based tracking coordinate (e.g., thefirst tracking coordinate 430 of FIG. 4 ) and a motion-based trackingcoordinate (e.g., the second tracking coordinate 440 of FIG. 4 ) are thesame, the processor 115 may select a grid region 600 having a squareshape. In this case, a length of one side of the grid region 600 may be2l₀. The value l₀ may be calculated through Equation 10.

$\begin{matrix}{l_{0} = {{- \frac{1}{\alpha}}\log c}} & \left\lbrack {{Equation}10} \right\rbrack\end{matrix}$

In an embodiment, when the image-based tracking coordinate and themotion-based tracking coordinate are different from each other, theprocessor 115 may select a grid region 610 having a rectangular shape.For example, when D is less than

$\frac{l_{0}}{\beta},$

the length of one side of the grid region 610 may be 2l₀ and the lengthof the other side may be 2l. In this case, the value l may be l₀−(1−β)D.In another example, when D is greater than or equal to

$\frac{l_{0}}{\beta},$

a length of one side of the grid region 610 maybe

$\frac{2\beta l}{{2\beta} - 1},$

and the length of the other side may be 2l.

FIG. 7 is a perspective view of an electronic apparatus 700 according toan embodiment.

Referring to FIG. 7 , the electronic apparatus 700 according to anembodiment may include a data acquisition unit 710 and a processor 720.The electronic device 700 may estimate a current pose of the electronicapparatus and predict a future pose of the electronic apparatus based onthe estimated current pose.

According to an embodiment, the electronic apparatus 700 may estimate asurrounding map of the electronic apparatus 700 and/or a current pose ofthe electronic apparatus 700 through simultaneous localization andmapping (SLAM).

In the disclosure, ‘SLAM’ may refer to a technique for acquiringinformation around an apparatus while moving in an arbitrary space, andestimating a map of the corresponding space and a current pose of theapparatus based on the acquired information, and the correspondingexpression may be used in the same meaning below.

For example, the processor 720 of the electronic apparatus 700 mayestimate a surrounding map and a current pose based on external data(e.g., image data, motion data, etc.) acquired through the dataacquisition unit 710.

In the disclosure, the pose of the electronic apparatus may denote dataincluding location information of the electronic apparatus, and theexpression may be used in the same meaning below. In this case, the posedata may include 6 DoF pose information, and the 6 DoF pose informationmay include information indicating a position and information indicatingan orientation of the electronic apparatus 700.

In an embodiment, the electronic apparatus 700 may be a wearableelectronic apparatus that may be worn on a part of the user's body. Forexample, the electronic apparatus 700 may further include a lens 730 anda connector 740 for fixing at least one region of the electronicapparatus 700 to a part of the user's body.

In an embodiment, the electronic apparatus 700 may be a glasses typewearable electronic apparatus that may be worn on a user's ear as shownin FIG. 7 , but is not limited thereto. In an embodiment, the electronicapparatus 700 may be a head mounted display (HMD) apparatus that may beworn on a user's head.

In an embodiment, the data acquisition unit 710 and the processor 720may be arranged in the connection unit 740, but the arrangementstructure of the data acquisition unit 710 and the processor 720 is notlimited thereto. In an embodiment, the data acquisition unit 710 and/orthe processor 720 may be arranged in a peripheral region (e.g., an edge)of the lens 730.

The electronic apparatus 700 may include optical components for emittinglight including data for an augmented reality image and controlling amovement path of the emitted light. The processor 720 may emit lightincluding data for the augmented reality image through the opticalcomponents, and cause the emitted light to reach the lens 730.

As the light including the data for the augmented reality image reachesthe lens 730, an augmented reality image may be displayed on the lens730, and the electronic apparatus 700 may provide the augmented realityimage to the user (or the wearer) through processes described above.

In FIG. 7 , only an embodiment in which the electronic apparatus 700 isa wearable electronic apparatus is illustrated, but an application fieldof the electronic apparatus (e.g., the electronic apparatus 100 of FIG.1 ) is not limited thereto. According to an embodiment, the electronicapparatus 100 may also be applied to an unmanned aerial vehicle (UAV)and/or an autonomous vehicle capable of estimating a surrounding map anda current pose thereof through SLAM.

Although the embodiments have been described in detail above, the scopeof the disclosure is not limited thereto, and various modifications andimprovements by those skilled in the art using the basic concept of thedisclosure as defined in the following claims are also within the scopeof examples and embodiments of the disclosure.

It should be understood that embodiments described herein should beconsidered in a descriptive sense only and not for purposes oflimitation. Descriptions of features or aspects within each embodimentshould typically be considered as available for other similar featuresor aspects in other embodiments. While one or more embodiments have beendescribed with reference to the figures, it will be understood by thoseof ordinary skill in the art that various changes in form and detailsmay be made therein without departing from the spirit and scope asdefined by the following claims.

What is claimed is:
 1. An apparatus for estimating an uncertainty, theapparatus comprising: a processor configured to estimate an uncertaintyof an image coordinate by executing at least one program, wherein theprocessor is further configured to: receive a first tracking coordinatethat corresponds to a reference coordinate, the reference coordinatebeing included in first image data acquired by a camera sensor, whereinthe first tracking coordinate comprises an image-based trackingcoordinate in second image data acquired after the first image data;acquire, based on motion data acquired from a motion sensor and a depthvalue of the first image data, a second tracking coordinate thatcorresponds to the reference coordinate, wherein the second trackingcoordinate comprises a motion-based tracking coordinate in the secondimage data; calculate a target coordinate distribution in the secondimage data based on the first tracking coordinate and the secondtracking coordinate; acquire an estimated target coordinate and anuncertainty of the estimated target coordinate based on the calculatedtarget coordinate distribution; and update the first tracking coordinatebased on the estimated target coordinate.
 2. The apparatus of claim 1,wherein the processor is further configured to: acquire a firstthree-dimensional ( 3D) coordinate with respect to the referencecoordinate based on the depth value of the first image data; convert thefirst 3D coordinate into a second 3D coordinate in the second image databased on the motion data; and acquire the second tracking coordinate byprojecting the second 3D coordinate.
 3. The apparatus of claim 1,wherein the processor is further configured to: for each of a pluralityof coordinates included in the second image data, calculate a weightedaverage of a first distance value and a second distance value, the firstdistance value being between the coordinate and the first trackingcoordinate and the second distance value being between the coordinateand the second tracking coordinate; for each of the plurality ofcoordinates, calculate a prior probability distribution based on thecalculated weighted average; determine a portion of the priorprobability distribution having a prior probability greater than orequal to a threshold value as a candidate coordinate group; andcalculate the target coordinate distribution based on the candidatecoordinate group.
 4. The apparatus of claim 3, wherein the processor isfurther configured to, for each of the plurality of coordinates, when avalue included in the motion data is less than a preset value, set aweight parameter corresponding to the first distance value to be greaterthan a weight parameter corresponding to the second tracking coordinate.5. The apparatus of claim 3, wherein the processor is further configuredto calculate an image-based distribution based on image similarity withthe reference coordinate for the determined candidate coordinate group.6. The apparatus of claim 5, wherein the processor is further configuredto calculate the target coordinate distribution based on the priorprobability distribution and the image-based distribution.
 7. Theapparatus of claim 1, wherein the processor is further configured totransmit the updated first tracking coordinate and the uncertainty ofthe estimated target coordinate to an external apparatus.
 8. Theapparatus of claim 1, wherein the processor is further configured toestimate a target coordinate by calculating an average of the targetcoordinate distribution, and estimate the uncertainty of the estimatedtarget coordinate by calculating a covariance matrix for the estimatedtarget coordinate.
 9. A method of estimating an uncertainty, the methodcomprising: receiving a first tracking coordinate that corresponds to areference coordinate, the reference coordinate being included in firstimage data acquired by a camera sensor, wherein the first trackingcoordinate comprises an image-based tracking coordinate in second imagedata acquired after the first image data; acquiring, based on motiondata acquired from a motion sensor and a depth value of the first imagedata, a second tracking coordinate that corresponds to the referencecoordinate, wherein the second tracking coordinate comprises amotion-based tracking coordinate in the second image data; calculating atarget coordinate distribution in the second image data based on thefirst tracking coordinate and the second tracking coordinate; acquiringan estimated target coordinate and an uncertainty of the estimatedtarget coordinate based on the calculated target coordinatedistribution; and updating the first tracking coordinate based on theestimated target coordinate.
 10. The method of claim 9, wherein theacquiring of the second tracking coordinate comprises: acquiring a first3D coordinate with respect to the reference coordinate based on thedepth value of the first image data; converting the 3D coordinate into asecond 3D coordinate in the second image data based on the motion data;and acquiring the second tracking coordinate by projecting the convertedsecond 3D coordinate.
 11. The method of claim 9, further comprising: foreach of a plurality of coordinates included in the second image data,calculating a weighted average of a first distance value and a seconddistance value, the first distance value being between the coordinateand the first tracking coordinate and the second distance value beingbetween the coordinate and the second tracking coordinate; for each ofthe plurality of coordinates, calculating a prior probabilitydistribution based on the calculated weighted average; determining, fromamong the prior probability distributions, coordinates from among theplurality of coordinates having a prior probability greater than orequal to a threshold value as a candidate coordinate group for theestimated target coordinate; and calculating the target coordinatedistribution based on the candidate coordinate group.
 12. The method ofclaim 11, further comprising calculating an image-based distribution forthe determined candidate coordinate group based on image similarity withthe reference coordinate.
 13. The method of claim 12, further comprisingcalculating the target coordinate distribution based on the priorprobability distribution and the image-based distribution.
 14. Themethod of claim 9, further comprising transmitting the updated firsttracking coordinate and the uncertainty of the estimated targetcoordinate to an external apparatus.
 15. The method of claim 9, furthercomprising estimating a target coordinate by calculating an average ofthe target coordinate distribution, and estimating the uncertainty ofthe estimated target coordinate by calculating a covariance matrix forthe estimated target coordinate.
 16. An electronic apparatus forperforming a simultaneous localization and mapping (SLAM) operation, theelectronic apparatus comprising: a camera sensor configured to acquireimage data about a surrounding environment; a motion sensor configuredto acquire motion data by detecting a rotation and a movement of theelectronic apparatus; and a processor electrically connected to thecamera sensor and the motion sensor, wherein the processor is configuredto: receive a first tracking coordinate that corresponds to a referencecoordinate, the reference coordinate being included in first image dataacquired by the camera sensor, the first tracking coordinate comprisingan image-based tracking coordinate in second image data acquired afterthe first image data; acquire, based on the motion data acquired fromthe motion sensor and a depth value of the first image data, a secondtracking coordinate that corresponds to the reference coordinate and isa motion-based tracking coordinate in the second image data calculate atarget coordinate distribution in the second image data based on thefirst tracking coordinate and the second tracking coordinate; acquire anestimated target coordinate and an uncertainty of the estimated targetcoordinate based on the calculated target coordinate distribution; andupdate the first tracking coordinate based on the estimated targetcoordinate.
 17. The electronic apparatus of claim 16, wherein theprocessor is further configured to: acquire a first 3D coordinate withrespect to the reference coordinate based on the depth value of thefirst image data; convert the first 3D coordinate into a second 3Dcoordinate in the second image data based on the motion data; andacquire the second tracking coordinate by projecting the convertedsecond 3D coordinate.
 18. The electronic apparatus of claim 16, furthercomprising a back-end processor configured to calculate a rotation angleand a position of the camera sensor, wherein the processor is furtherconfigured to transmit the updated first tracking coordinate and theuncertainty of the estimated target coordinate to the back-endprocessor.
 19. The electronic apparatus of claim 16, wherein theprocessor is further configured to estimate a target coordinate bycalculating an average of the target coordinate distribution, andestimate the uncertainty of the estimated target coordinate bycalculating a covariance matrix with respect to the estimated targetcoordinate.
 20. The apparatus of claim 1, wherein the processor isfurther configured to display an image corresponding to the updatedfirst tracking coordinate.